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Abstract 

This  study  explores  the  relationship  between  an  organization’s  technology  development 
approach  (TDA)  and  its  success  in  winning  SBIR  Phase  1  competitions.  The  research  posits 
a  difference  in  SBIR  Phase  1  success  between  companies  using  sustaining  and  disruptive 
TDA.  Companies  responding  to  DARPA  SBIR  Phase  1  solicitations  are  surveyed  to 
determine  their  approach  to  developing  technology  solutions.  A  newly  developed  instrument 
is  used  to  identify  disruptive  and  sustaining  TDA.  Demographic  data,  SBIR  success,  and  data 
on  TDA  are  collected.  Principal  axis  factoring  was  used  to  identify  four  orthogonal  factors 
related  to  TDA.  The  factors  were  interpreted  and  a  new  objective  scale  was  developed  to 
measure  sustaining  TDA.  No  equivalent  scale  was  developed  for  disruptive  TDA.  Correlation 
and  regression  analyses  indicated  TDA  is  a  contributing  factor  in  SBIR  Phase  1  success. 
Focus  on  the  defense  market  and  company  inventiveness  as  measured  by  patent  activity  are 
also  predictors.  The  findings  indicate  TDA  mediates  a  company’s  focus  and  success  at 
winning  SBIR  Phase  1  contracts.  Implications  for  defense  acquisition  and  disruptive 
innovation  are  discussed.  A  new  conceptual  framework  linking  focus,  capability, 
inventiveness,  and  TDA  with  SBIR  Phase  1  success  is  proposed  for  further  research. 

Introduction 

Since  the  end  of  World  War  II,  advanced  technology  has  been  the  foundation  of  U.S. 
military  strategic  advantage.  The  Department  of  Defense  (DoD)  consistently  invests  $60 
billion  to  $80  billion  per  year  on  weapons  system  research  and  development  to  maintain  and 
increase  the  technology  advantage  (USD(C),  2013;  Watts,  2008).  These  investments  dwarf 
similar  investments  of  other  industrialized  nations;  Canada,  for  example,  projects  R&D, 
procurement,  and  operations  investment  over  20  years  equivalent  to  a  single  year’s  U.S. 
defense  budget  (Jenkins,  Castelli,  Cianfarani,  Fraser,  &  Nicholson,  2013). 

The  United  States  is  arguably  second-to-none  in  aggressively  harvesting  the  best 
technologies  for  incorporation  into  weapons  systems,  but  it  comes  at  a  very  high  price.  Most 
of  the  investment  since  World  War  II  has  been  used  to  refine  and  produce  technologies  that 
existed  prior  to  that  conflict  (Guilmartin,  1994).  The  conceptual  bases  of  nuclear  weapons 
and  radar  were  understood  in  the  1920s  and  late  1930s.  Rocketry,  jet  propulsion,  radar,  and 
aircraft  advances  were  driven  by  the  focused  application  of  improved  production  and  design 
capabilities.  The  Cold  War  was  the  evolutionary  pressure  for  developing  the  entire  nuclear 
industrial  base.  The  R&D  focus  since  that  time,  enabled  by  the  unique  structure  of  the  U.S. 
defense  industrial  base,  has  encouraged  the  continued  emphasis  on  sustaining 
technologies  while  giving  less  emphasis  to  the  systematic  encouragement  and  support  of 
disruptive  technologies. 

Disruptive  Innovation 

Disruptive  technologies  and  scenarios  have  defining  attributes  (Christensen,  1997). 
Disruptive  technologies  are  existing  technologies  configured  in  novel  ways  or  placed  in  an 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-364- 


entirely  new  architecture.  They  typically  have  lower  performance  in  the  primary  performance 
dimension,  but  they  often  are  cheaper  to  implement  or  operate.  They  usually  have 
performance  in  secondary  areas  that  make  them  attractive  to  an  underserved,  or  new, 
market.  Finally,  their  value  network,  or  supporting  supplier  and  talent  infrastructure,  is 
smaller  and  cheaper  than  the  value  network  of  the  legacy  technology. 

The  disruptive  scenario  is  usually  one  of  the  disruptive  technology  making  inroads 
into  the  legacy  technology  market  share  at  the  low  end  of  a  market.  The  cheaper  value 
networks  of  the  disruptive  technology,  coupled  with  attractive  secondary  performance 
characteristics,  represents  a  good  value  proposition  to  the  segment  of  the  market  that 
cannot  afford  the  market-leading  technology.  Christensen  (1997)  built  this  scenario  from  the 
history  of  the  computer  disk  drive  market.  As  computers  became  smaller,  so  did  the  memory 
disk  drives  offered  by  new  entrants  to  the  drive  market.  These  new  entrants  produced  8” 
disk  drives  for  the  emergent  mini-computer  market  and,  in  the  process,  acquired  market 
share  from  legacy  disk  drive  vendors  who  had  previously  been  successful  in  producing 
larger  drives. 

This  scenario  repeated  itself  with  the  advent  of  the  personal  computers.  A  new  set  of 
disk  drive  vendors  successfully  marketed  the  5  14”  and  3  14”  drives  and  eventually  displaced 
the  8”  drive  vendors.  In  each  case,  the  disruptive  disk  drive  was  developed  from  existing 
technology,  initially  had  diminished  performance  compared  to  the  legacy  technology,  and 
had  attractive  secondary  attributes  for  a  new  or  underserved  market.  While  cost  per 
megabyte  of  storage  was  higher  than  for  legacy  drives,  the  form  factor  was  an  attractive 
feature  for  the  emerging  personal  computer  market. 

Disruptive  innovation  also  operates  on  the  scale  of  an  entire  market.  The  story  of 
Eastman  Kodak  reconfigured  the  architecture  of  the  entire  photography  market  by 
introducing  an  existing,  but  unused,  film  technology  into  the  camera  itself  and  altering  the 
nature  of  the  photofinishing  process  (Lucas  Jr.  &  Goh,  2009;  Utterback,  1995).  George 
Eastman  replaced  the  emulsion-covered  glass  plates  of  the  large  format  cameras  with  a 
flexible  film  that  could  be  rolled  up  into  a  cheaply  produced  box  camera — the  “Brownie.”  His 
company  then  assumed  the  task  of  processing  the  exposed  film,  a  business  move  that 
opened  up  the  field  of  photography  to  average  consumers.  This  change  in  the  business 
architecture  yielded  a  tremendously  attractive  performance  advantage  for  the  box  camera. 
While  a  large  format  view  camera  would  always  take  superior  quality  pictures,  it  was  only 
available  to  those  with  expertise  in,  and  desire  to,  chemically  process  the  film.  The  Kodak 
box  camera  took  lower  quality  photos,  but  was  very  accessible  to  the  casual  hobbyist.  Over 
time,  the  quality  of  “casual  photo”  pictures  improved,  driving  large  format  photography  into  a 
niche  market. 

A  century  later  the  scenario  repeated  itself  in  amateur  photography.  Kodak  had 
become  locked  into  their  century-old  business  model  to  the  point  where  they  saw 
themselves  as  a  chemical  and  film  company  instead  of  a  company  dealing  in  imaging 
(Lucas  &  Goh,  2009;  Utterback,  1995).  Despite  owning  the  seminal  patents  for  digital 
photography,  Kodak  was  slow  to  develop  a  digital  camera  and  accompanying  business 
model  for  the  digital  imaging  market.  Computer  and  technology  companies  such  as  Sony, 
Canon,  and  Hewlett-Packard  established  an  early  and  persistent  presence  in  the  market. 
After  the  mid-1990s,  Kodak  was  unable  to  carve  out  a  profitable  niche  and  is  currently  in 
bankruptcy,  while  still  making  film  for  specialized  markets. 

Disruptive  Innovation  and  Defense  Acquisition 

Literature  in  defense  acquisition  emphasizes  the  need  for  developing  disruptive 
technologies,  but  often  the  term  is  misused  when  compared  to  its  classical  definition. 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-365- 


Reference  to  disruptive  technology  in  a  military  context  is  often  confused  with  any  new 
technology.  The  disruptive  scenario  in  the  military  acquisition  literature  casts  disruption  as 
neutralizing  or  frustrating  an  adversary’s  strategic  or  tactical  advantage  (Johnson  & 
McLaughlin,  2007;  Shannon,  Yarbrough,  &  Johnson,  2010).  Consequently  the  searches  for 
what  are  called  disruptive  technologies  and  innovations  usually  address  improving 
sustaining  technologies  in  already  existing  architectures.  This  fundamental 
misunderstanding  of  disruptive  innovation  in  the  military  context  hampers  the  search  for  truly 
disruptive  architectures. 

The  famed  British  Admiral  Sir  John  Jellicoe  was  frustrated  by  the  relative 
ineffectiveness  of  British  surface  combatants  in  World  War  I,  even  though  in  the 
Dreadnought  class  of  battleships  Britain  and  the  Allies  had  very  advanced  technology  in 
steam  turbine  propulsion  and  advanced  fire  control.  Equivalent  technology  today  would  be 
termed  disruptive,  but  in  reality  these  advances  were  extensions  of  existing  technologies. 
The  technology  was  not  only  easily  matched  by  adversaries,  but  was  ineffective  for  either 
side,  in  part  because  the  command  and  control  architecture  of  naval  surface  combat  had 
changed  little  since  the  Napoleonic  wars  (Brooks,  2007;  Hoffman,  2004;  Mukunda,  2010). 

Naval  warfare  in  World  War  I  experienced  disruption,  but  not  because  of  any 
advanced  technology.  The  true  disruption  was  in  the  use  of  submarines  against  Allied 
shipping  by  Germany  and  the  anti-submarine  tactics  response  of  the  Allies.  Germany 
changed  the  architecture  of  naval  warfare  by  ceding  the  high  seas  to  the  British  surface  fleet 
and  preying  on  transatlantic  commercial  shipping.  This  tactic  was  very  effective  for  the  first 
two  years  of  the  conflict,  in  part  because  the  British  Admiralty  did  not  view  protection  of 
commercial  shipping  as  part  of  its  mission.  In  the  parlance  of  disruptive  innovation  theory, 
this  was  not  part  of  the  “market”  in  which  it  wished  to  engage.  By  using  the  submarine,  an 
existing  technology,  in  a  new  role  Germany  reconfigured  the  architecture  of  naval  warfare. 
After  massive  shipping  losses,  the  Allies  responded  to  the  new  architecture  by  adapting  anti¬ 
submarine  tactics  and  organizing  transatlantic  convoys  for  commercial  shipping.  It  is 
interesting  to  note  that  the  Allied  response  also  used  existing  technologies,  again  in  a 
different  role. 

After  World  War  II  the  architecture  of  warfare  emphasized  production  efficiency  and 
operational  maneuver  (Guilmartin,  1994).  The  acquisition  policies  and  market  forces,  since 
the  1950s,  have  favored  and  promoted  sustaining  technology  development  to  fit  this 
emphasis.  The  implicit  assumption  has  been  that  the  outcome  of  future  conflicts  would  again 
be  dependent  on  production  and  operational  maneuver  (Guilmartin,  1994;  Watts,  2008; 
Watts  &  Harrison,  2011).  The  recent  conflicts  in  Iraq  and  Afghanistan  signal  that  this  warfare 
architecture  may  be  about  to  change. 

This  trend  of  searching  for  disruptive  innovations,  while  actually  focusing  on  and 
funding  sustaining  technologies  is  becoming  increasingly  expensive  and  arguably 
unsustainable  (GAO,  2005,  2006,  2006a,  2006b).  Supporting  sustaining  technologies  is 
critical  and  necessary  to  maintaining  the  U.S.  technological  edge;  however,  identifying  and 
nurturing  disruptive  innovation  in  the  classic  sense  of  the  term  could  arguably  yield  solutions 
that  are  truly  innovative  and  at  a  lower  cost.  It  is  not  an  either-or  choice,  but  rather  one  of 
balance  in  efforts,  resources,  and  focus.  This  study  begins  to  address  the  focus  by 
characterizing  the  technology  development  approach  (TDA)  of  a  company  as  either 
intrinsically  sustaining  or  disruptive,  and  assessing  whether  their  technology  solutions  are 
the  preferred  ones  in  an  acquisition  competition. 
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Significance  of  the  Study 

This  study  examined  the  technology  development  approach  (TDA)  of  small 
innovative  companies  and  established  a  quantitative  relationship  between  TDA  and  success 
in  defense  acquisition  contracting.  Establishing  such  a  relationship  is  a  key  indicator  of 
whether  a  company’s  TDA  addresses  a  disruptive  or  sustaining  focus  in  technology 
development  in  a  deliberate  and  systematic  way.  Establishing  metrics  for  sustaining  and 
disruptive  TDA  would  give  defense  R&D  professionals  a  significant  tool  to  assess  whether  a 
company  has  the  potential  for  producing  disruptive  technology  solutions. 

The  results  also  fill  a  gap  in  the  development  of  disruptive  innovation  theory,  that  is, 
the  lack  of  predictive  effectiveness  in  the  theory  itself  (Danneels,  2004,  2006;  Markides, 
2006;  Tellis,  2006).  A  known  weakness  in  disruptive  innovation  theory  lies  in  the 
terminology.  The  term  “disruptive”  is  used  to  describe  an  attribute  of  the  technology  as  well 
as  a  technology  solutions  effect  on  its  market.  As  with  defense  acquisition,  establishing 
objective  metrics  for  sustaining  and  disruptive  TDA  begins  to  separate  the  technology 
attributes  from  the  market  effects. 

Study  Overview 

This  was  a  non-experimental,  exploratory  study  designed  to  identify  disruptive  and 
sustaining  TDA  in  small  innovative  companies.  Companies  participating  in  the  U.S. 
government  Small  Business  Innovative  Research/Small  Business  Technology  Transfer 
(SBIR/STTR)  program  were  sampled  with  a  new  instrument  designed  to  measure  disruptive 
and  sustaining  TDA  (DoD,  2013).  The  instrument  collected  primary  data  on  company 
demographics  such  as  total  revenues,  company  age,  number  of  employees,  and  so  forth.  It 
also  measured  company  TDA  preference  through  12  Likert-style  statements  developed  from 
disruptive  innovation  theory  and  systems  engineering  practices. 

The  study  design  posited  that  TDA  would  be  a  primary  driver  for  success  in 
competing  for  SBIR/STTR  contracts.  The  initial  concept  framework  is  at  Figure  1. 


Figure  1.  Initial  Study  Concept  Diagram 

The  TDA,  either  disruptive  or  sustaining  in  nature,  would  be  mediated  by  a 
company’s  focus,  capacity,  invention  or  inventiveness,  and  experience.  Company  focus  was 
measured  by  a  company’s  revenue  from  defense  contracts.  Inventiveness  was  measured  by 
the  patent  activity  of  a  company.  Capacity  related  to  the  scientific  and  technical  staff 
employed  by  a  company,  and  the  age  of  a  company  was  the  measure  for  experience. 

The  primary  variable  TDA  is  a  new  construct  derived  from  responses  to  the  Likert- 
style  statements  of  the  survey  instrument.  The  survey  responses  were  subjected  to  a 
dimension  reduction  and  potential  measurement  scales  for  sustaining  and  disruptive  TDA 
were  developed.  With  a  quantified  measure  of  TDA,  the  demographic  data  were  subjected 
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to  a  multivariate  regression  and  correlation  analysis  to  determine  the  relative  contributions  of 
the  independent  variables. 

Research  Questions  and  Hypotheses 

This  study  poses  two  research  questions  and  associated  hypotheses: 


RQi:  What  is  the  relationship  among  the  independent  variables  of  TDA, 
company  demographics,  and  the  criterion  variable  of  winning  Phase  1  SBIR 
defense  contracts? 

RQ2:  What  are  the  relative  contributions  of  TDA  and  company  demographics 
to  winning  SBIR  Phase  1  contracts? 


The  first  research  question  is  designed  to  identify  any  relationship  among  the  way 
companies  develop  their  technology  solutions,  the  demographic  profile  of  the  company,  and 
the  criterion  variable  of  winning  SBIR  Phase  1  contracts.  The  second  research  question 
attempts  to  quantify  the  identified  relationships  of  RQi. 

The  aim  of  the  study  is  to  examine  four  hypotheses  in  support  of  answering  the 
research  questions: 

Hoi  There  is  no  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

Hi  1  There  is  a  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

Ho2  There  is  no  significant  relationship  between  TDA  and  the  contribution  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts. 

Hi2  There  is  a  significant  relationship  between  TDA  and  the  contributions  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts. 

Hypotheses  Hoi  and  Hi  1  are  designed  to  determine  whether  disruptive  or  sustaining 
TDA  are  related  to  success  in  SBIR  Phase  1  competitions.  The  Hypotheses  Ho2  and  Hi2 
test  for  the  significance  of  the  relationship  of  TDA  with  the  demographic  variables  of  the 
company. 

Population/Sample 

The  target  population  was  small  innovative  companies  engaged  in  developing 
technology  solutions.  Companies  participating  in  the  SBIR/STTR  program  were  used  as  a 
surrogate  for  this  population.  The  regulations  for  company  participation  stipulated  American 
companies  with  a  staffing  of  less  than  500  (DoD,  2013).  The  research  focused  on 
companies  competing  for  SBIR  Phase  1  contracts.  Technologies  at  the  Phase  1  level 
aligned  well  with  a  key  attribute  of  disruptive  technologies,  that  is,  technologies  that  have  yet 
to  develop  a  significant  value  network  providing  institutional  or  market  support  (Christensen, 
1997;  Christensen  &  Raynor,  2003). 

The  sampling  frame  was  developed  from  companies  responding  to  Defense 
Advanced  Research  Project  Agency  (DARPA)  SBIR  Phase  1  solicitations  between  2007 
and  2013.  This  yielded  a  sampling  frame  of  462  companies.  Companies  were  randomly 
selected  from  the  sampling  frame  to  participate  in  the  study.  Organizations  signaling  interest 
were  sent  an  information  package  detailing  the  study  and  a  copy  of  the  survey  instrument. 
Companies  wishing  to  participate  were  sent  an  email  invitation  given  them  access  to  the 
survey  instrument  hosted  on  the  SurveyMonkey.com  commercial  website. 
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Survey  Instrument  Validation  and  Data  Collection 

The  survey  instrument  was  specially  developed  for  this  study.  Consequently  no 
validity  or  reliability  data  were  available.  Field  and  pilot  tests  were  conducted  to  confirm  the 
survey’s  usability  and  its  reliability.  Practitioners  in  program  management  and  industrial  base 
analysts  reviewed  the  survey  instrument  in  a  field  test  and  suggested  changes  in  wording  to 
improve  readability. 

The  pilot  test  involved  responses  from  nine  companies  selected  at  random  from  the 
sampling  frame.  Of  particular  interest  were  the  12  Likert-style  survey  items  measuring  TDA. 
These  measurements  would  form  the  basis  for  scales  measuring  disruptive  and  sustaining 
TDA.  Table  1  lists  pilot  test  results  with  respect  to  scale  reliability  of  the  12  statements 
measuring  TDA. 


Table  1.  Item  Deletion  Effects  on  Cronbach’s  Alpha 


Scale  Mean  if  Item 
Deleted 

Scale  Variance  if 
Item  Deleted 

Corrected  Item- 
Total  Correlation 

Cronbach’s  Alpha 
if  Item  Deleted 

TDA15 

38.71 

51.238 

.507 

.858 

TDA16 

38.00 

44.667 

.784 

.837 

TDA17 

38.14 

44.476 

.717 

.842 

TDA18 

37.86 

51.810 

.226 

.882 

TDA19 

38.00 

44.667 

.596 

.854 

TDA20 

37.57 

48.286 

.634 

.849 

TDA21 

37.86 

40.476 

.956 

.820 

TDA22 

37.57 

51.619 

.815 

.851 

TDA23 

38.00 

54.000 

.270 

.869 

TDA24 

37.00 

55.333 

.148 

.875 

TDA25 

38.14 

53.810 

.325 

.866 

TDA26 

37.29 

47.238 

.861 

.838 

The  12  items,  TDA15-TDA26,  displayed  high  values  of  Cronbach  alpha  when  taken 
as  a  composite  scale.  Deleting  any  one  item  from  the  scale  had  minimal  effect  on  the  scale 
validity  and  only  depressed  the  scale  to  0.820.  This  is  well  above  the  Cronbach  alpha 
standard  of  0.7  for  valid  and  reliable  scales.  No  changes  were  made  to  the  survey 
instrument  after  the  pilot  test  and  the  research  moved  on  to  the  full-scale  study. 

A  total  of  70  companies  were  randomly  selected  and  contacted  from  the  original 
sampling  frame  of  420  organizations.  Of  these  70  companies,  36  expressed  interest  and 
requested  further  information.  Study  details  and  a  copy  of  the  survey  were  sent  with  a 
secure  email  link  to  the  survey  instrument.  A  total  of  18  companies  completed  the  survey, 
yielding  16  usable  responses.  The  final  response  rate  for  the  full-scale  study  was  22.8%. 

Factor  Analysis  and  Scale  Development 

Data  analysis  was  performed  in  a  specific  sequence.  Descriptive  statistics  were 
calculated  on  the  demographic  and  TDA  responses.  Factors  from  the  TDA  data  were 
extracted,  retained,  and  interpreted.  Potential  scales  for  disruptive  and  sustaining  TDA  were 
developed.  Finally,  new  independent  variables  for  TDA  were  developed  from  the  scales  and, 
along  with  the  collected  demographic  data,  were  regressed  against  a  company’s  success  in 
winning  SBIR  Phase  1  awards. 

Factor  Extraction 

The  12  survey  items  addressing  TDA  were  designed  to  measure  the  level  of 
sustaining  or  disruptive  TDA  of  each  of  the  participating  companies.  The  Likert-style 
responses  were  dimensionally  reduced  via  principal  axis  factoring  and  yielded  12  factors. 
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Scree  Plot 


Figure  2.  Scree  Plot  of  Extracted  Factors 

The  first  four  factors  accounted  for  -75%  of  the  variance  in  the  TDA  data.  Factors  5- 
12  displayed  relatively  little  contribution  and  were  judged  to  be  not  significant  in  developing 
metrics  for  TDA.  The  four  strongest  factors  were  subsequently  retained  for  further  analysis. 

The  four  factor  solutions  were  rotated  with  a  Direct  Oblimin  transformation  to  obtain 
optimized  loading  of  the  contributing  survey  items  on  the  identified  factors.  The  structure 
matrix  identifies  the  correlation  of  the  survey  items  with  the  four  retained  factors. 

Table  2.  Structure  Matrix  for  TDA 


Factor 


1 

2 

3 

4 

TDA16 

.813 

.118 

- 

.074 

.537 

TDA21 

.774 

-.139 

- 

.236 

.500 

TDA17 

.716 

-.243 

.037 

TDA15 

.617 

-.264 

- 

.325 

- 

.238 

TDA19 

.508 

-.411 

- 

.463 

.248 

TDA26 

.725 

-.783 

.223 

- 

.093 

TDA23 

.363 

-.708 

.150 

.045 

TDA22 

.017 

-.607 

.211 

- 

.149 

TDA20 

.024 

-.529 

.388 

- 

.027 

TDA24 

.150 

.091 

.710 

.157 

TDA18 

.341 

.176 

.649 

.096 

TDA25 

.062 

.063 

.134 

.793 

Extraction  Method:  Principal  Axis  Factoring 
Rotation  Method:  Oblimin  with  Kaiser  Normalization 
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Factor  Loading 

The  survey  items  highlighted  in  Table  2  are  the  highest  loading  survey  items  on  the 
four  retained  factors.  Best  practice  with  factor  analysis  is  to  consider  loading  item  loadings 
above  an  absolute  value  of  0.3  to  be  significant  for  factor  interpretation^.  S.  Cooper,  2003; 
Costello  &  Osborne,  2005;  Gorsuch,  1997).  At  least  four  survey  items  load  at  this  or  higher 
level  on  each  of  the  retained  factors.  Each  factor  has  enough  items  to  form  workable  scales 
for  establishing  TDA,  subject  to  interpretation  of  the  factor  matrix. 

Factor  Interpretation  and  Scale  Reliability 

The  highest  loading  survey  items  for  Factor  1  are  characteristic  of  sustaining  TDA. 
Factor  2  loading  was  associated  with  survey  items  describing  technology  architectures  or 
market  effects.  Factor  3  displays  survey  items  with  attributes  of  both  sustaining  and 
disruptive  TDA  as  well  as  significant  cross-loading  with  Factors  1  and  2.  The  items  loading 
strongly  on  Factor  4  were  mostly  redundant  with  Factor  1. 

Sustaining  TDA  was  well  described  by  Factor  1  while  Factor  2  showed  promise  as  a 
construct  for  disruptive  TDA.  The  other  two  factors  were  ambiguous;  however,  scales  were 
constructed  and  analyzed  to  determine  the  validity  and  suitability  for  including  any  of  the 
factors  in  the  following  regression  analysis.  Table  3  lists  the  Cronbach  alpha  for  each  of  the 
four  factor  scales  constructed  from  the  contributing  survey  items.  Table  4  is  an  analysis  of 
the  robustness  of  each  scale. 

Table  3.  Cronbach  Alpha  for  Prospective  Factor  Scales 

Cronbach’s  Alpha  Based  on 

Factor  Cronbach’s  Alpha_ Standardized  Items 


1 

.833 

.833 

2 

.727 

.730 

3 

.448 

.522 

4 

.762 

.752 

N  =  4  scale  items  for  each  factor 


Table  4.  Scale  Robustness 


Factor 

Cronbach’s  Alpha 

Cronbach  Alpha  Range 

Item  Deletion  Sensitivity 

1 

.833 

.757  -  .832 

0 

2 

.727 

.601  -  .753 

3 

3 

.448 

.227  -  .526 

n/a 

4 

.762 

.532  -  .803 

2 

Three  of  the  four  factors  support  scales  with  acceptable  alpha  values  of  0.7, 
suggesting  good  reliability  with  scale  constituents  that  are  complementary  without  being 
redundant  (Gliem  &  Gliem,  2003;  Gorsuch,  1997;  Tavakol  &  Dennick,  2011).  The  scale  for 
Factor  3  falls  well  below  this  level,  making  the  factor  unsuitable  for  inclusion  in  further 
analysis. 

The  robustness  of  the  scales  is  measured  by  the  effect  of  item  deletion.  The  scale  for 
Factor  1  maintains  acceptable  alpha  values  regardless  of  item  deletion.  Scales  for  Factors  2 
and  4  indicate  fragile  reliability;  alpha  values  fall  when  at  least  two  items  of  either  scale  are 
deleted. 

Factor  Retention 

Factor  1  is  retained  for  further  analysis,  based  on  the  homogeneity  of  the  component 
survey  items  and  the  robustness  of  the  resultant  scale.  The  scale  from  Factor  1  is  measures 
the  construct  sTDA ,  an  independent  variable  used  as  a  measure  of  sustaining  TDA.  Factor 
2  is  rejected  because  of  its  heterogeneous  mix  of  survey  items  and  fragility  of  the  resultant 
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scale  (D.  R.  Cooper  &  Schindler,  2011;  Gorsuch,  1997;  Vogt,  2007).  Factors  3  and  4  are 
rejected  due  to  inadequate  scales  in  the  case  of  the  former  and  redundancy  with  Factor  1  in 
the  case  of  the  latter. 

Research  Questions  and  Associated  Hypotheses 

The  research  questions  were  addressed  with  a  mix  of  correlation  and  regression 
analyses.  The  questions  RQi  and  the  associated  hypotheses  Hoi  and  Hi  1  explore  the 
quantitative  relationships  among  sTDA,  the  collected  demographic  data,  and  success  at 
winning  SBIR  Phase  1  contracts.  The  question  RQ2  and  associated  hypotheses  H02  and 
Hi2  determine  the  contributions  of  the  same  independent  variables  and  demographics  in 
winning  SBIR  Phase  1  contracts. 

Research  Question  1  and  Hypotheses  Hol/Hil 

Research  Question  1  addresses  the  relationships  among  TDA,  company 
demographics,  and  SBIR  Phase  1  success: 

RQi  What  is  the  relationship  among  the  independent  variables  of  TDA, 
company  demographics,  and  the  criterion  variable  of  winning  Phase  1  SBIR 
defense  contracts? 

Correlation  among  sTDA  and  the  demographic  variables  successfully  addresses  this 
question.  Table  5  contains  a  description  of  the  demographic  variables  and  the  construct 
sTDA  and  Table  6  is  the  resulting  correlation  matrix. 

Table  5.  Demographic  Variable  Definition 


Variable  Name_ Definition_ Data  Type 


EXP01 

Company  age  (years) 

ratio 

CAP02 

Total  employment 

ratio 

CAP03 

Total  Science  &  Technical  Staff 

ratio 

EXP04 

SBIR  Phase  1  Applications 

ratio 

EXP05 

SBIR  Phasel  Awards 

ratio 

INV06 

Patent  applications 

ratio 

INV07 

Patents  granted 

ratio 

FOC08 

Total  annual  revenue 

ratio 

FOC09 

SBIR  Phase  1  &  2  revenue 

ratio 

FOC10 

Total  defense  revenue 

ratio 

sTDA 

Sustaining  Technology  Development  Approach 

ratio 

Table  6. 

Correlation  Matrix  for  Predictor  and  Criterion  Variables 

EXP01 

CAP02 

CAP03 

EXP04 

EXP05 

INV06 

INV07 

FOC08 

FOC09 

FOC10  sTDA 

EXP01 

1 

CAP02 

.174 

1 

CAP03 

.420 

.179 

1 

EXP04 

.474* 

.074 

.791" 

1 

EXP05 

.626" 

.253 

.776** 

.802** 

1 

INV06 

.351 

-.027 

.568* 

.753" 

.601" 

1 

INV07 

.315 

-.004 

.621** 

.601** 

.441* 

.807" 

1 

FOC08 

.291 

.407 

.623" 

.701" 

.581" 

.588** 

.520* 

1 

FOC09 

.405 

.374 

.678** 

.351 

.737** 

.237 

.218 

.285 

1 

FOC10 

.266 

.435* 

.479* 

.693" 

.547* 

.579** 

.496* 

.896** 

.271 

1 

sTDA 

-.137 

.336 

.297 

.316 

.353 

.067 

.120 

.431* 

.219 

.367  1 

*  Correlation  is  significant  at  the  0.05  level  (1 -tailed). 
**  Correlation  is  significant  at  the  0.01  level  (1-tailed). 
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The  criterion  variable  EXP05  is  strongly  and  significantly  correlated  with  all  the 
demographic  variables  except  for  CAP02,  and  displays  a  weak  correlation  with  sTDA,  the 
construct  for  sustaining  technology  development. 

The  Hypotheses  Hoi  and  Hi  1  were  designed  to  measure  the  difference  in  sustaining 
and  disruptive  technology  development  approaches  in  SBIR  Phase  1  competitions. 

Hoi  There  is  no  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

Hil  There  is  a  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

While  a  metric  for  sustaining  TDA  was  established  through  factor  analysis  and  scale 
development,  no  analogous  measure  could  be  developed  for  disruptive  TDA.  A  factor  was 
extracted  with  attributes  of  disruptive  TDA;  however,  its  scale  reliability  was  inadequate  to 
properly  develop  a  reliable  construct.  Consequently,  the  research  could  only  measure 
sustaining  TDA. 

Research  Question  2  and  Hypotheses  Ho2/Hi2 

Research  question  RQ2  attempts  to  quantify  the  contribution  of  company 
demographics  and  TDA  to  SBIR  Phase  1  success.  The  hypotheses  Ho2  and  Hi2  are 
designed  to  determine  whether  those  contributions  are  significant. 

RQ2  What  are  the  relative  contributions  of  TDA,  and  company  demographics 
to  winning  SBIR  Phase  1  contracts? 

Ho2  There  is  no  significant  relationship  between  TDA  and  the  contribution  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts. 

Hi2  There  is  a  significant  relationship  between  TDA  and  the  contributions  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts 

This  study  regressed  the  number  of  SBIR  Phase  1  successes  against  the 
demographic  data  and  sTDA  to  determine  regression  coefficients  in  a  standard  linear 
regression  model: 

EXPOS  =  a  +  B1  (sTDA)  +  B2(EXP01)  +  B3(CAP03 )  +  B4(INV06)  +  B5(FOC09)  +  e  (1) 

The  variables  used  in  the  regression  analysis  are  listed  in  Table  7. 


Table  7.  Regression  Analysis  Variable  Definition 


Variable 

Type 

Definition 

EXP05 

Dependent 

SBIR  Phase  1  awards 

sTDA 

Independent 

Sustaining  TDA 

EXP01 

Independent 

Company  age 

CAP03 

Independent 

Science  and  technical  staff 

INV06 

Independent 

Patent  applications 

FOC09 

Independent 

Revenue  from  SBIR  Phase  1  and  2  award 

The  variables  chosen  for  analysis  are  a  subset  of  the  survey  data  and  represent  the 
constructs  of  focus,  inventiveness,  experience,  capacity,  TDA,  and  SBIR  success  described 
in  the  concept  diagram  of  Figure  1.  Tables  8  and  9  summarize  the  results  of  the  regression 
analysis. 
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Table  8.  Regression  Analysis  Summary 

Change  Statistics 


Adjusted 

SE  of  the 

R2 

F 

Sig.  F 

R2 

R2 

Estimate 

Change 

Change 

dfl 

df2 

Change 

869 

.803 

17.654 

.869 

13.240 

5 

10 

.000 

Predictors:  (Constant),  FOC09,  STDA,  INV06,  EXP01,  CAP03 
Dependent  Variable:  EXP05 


Table  9.  Regression  Analysis  Coefficients 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

Model 

B 

Std.  Error 

Beta 

t 

Sig. 

(Constant) 

-53.001 

20.952 

-2.530 

.030 

STDA 

2.742 

1.334 

.260 

2.056 

.067 

EXP01 

1.362 

.535 

.349 

2.545 

.029 

CAP03 

.645 

1.085 

.117 

.594 

.566 

INV06 

1.821 

.886 

.302 

2.056 

.067 

FOC09 

6.888E-006 

.000 

.388 

2.344 

.041 

Dependent  Variable:  EXP05 

The  regression  model  explains  -87%  of  the  variance  in  the  criterion  variable  of 
success  in  winning  SBIR  Phase  1  contracts.  Company  age  and  experience,  EXP01  and 
FOC09,  respectively  are  the  strongest  predictors  of  SBIR  Phase  1  success,  followed  by  the 
variables  for  inventiveness  and  TDA,  INV06  and  STDA,  respectively.  The  strongest 
predictors  are  significant  at  p  <  0.05  level.  The  coefficients  for  STDA  and  INV06  are  slightly 
less  significant,  each  with  a  value  of  p  =  0.067.  The  construct  for  capacity,  that  is,  the 
number  of  science  and  technical  staff  represented  by  CAP03  appears  to  have  the  weakest 
predictive  value  in  the  regression  model. 

Conclusions 

Both  research  questions  and  one  set  of  hypotheses  were  successfully  addressed  in 
this  study.  The  data  successfully  addressed  RQi  and  established  a  quantifiable  relationship 
among  the  demographic  variables,  TDA,  and  success  in  competing  for  SBIR  Phase  1 
contracts.  The  collected  data  could  not  address  the  Hypotheses  Hoi  and  Hi  1  designed  to 
discern  a  significant  difference  in  effectiveness  in  disruptive  or  sustaining  TDA.  This  was  due 
to  the  fact  that  disruptive  TDA  could  not  be  reliably  measured.  A  reliable  measurement  for 
sustaining  TDA,  sTDA,  was  established  and  used  in  the  analyses  for  RQ2  and  Hypotheses 
H02andHi2. 

The  demographic  variables  representing  company  experience,  focus,  inventiveness, 
capacity,  and  TDA  were  found  to  be  significant  contributors  to  the  success  of  a  company  in 
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competing  for  SBIR  Phase  1  contracts.  The  linear  regression  model  indicates  that  company 
age  and  focus  on  defense  projects  proved  to  be  the  strongest  predictors  of  SBIR  Phase  1 
success.  Inventiveness  and  a  sustaining  TDA  were  also  significant  predictors,  although  at  a 
lower  level  than  experience  and  focus.  Capacity,  as  measured  by  the  number  of  science 
and  technical  staff,  was  found  to  have  minimal,  significant  contribution  to  SBIR  Phase  1 
success.  The  results  support  the  initial  conceptual  framework  of  Figure  1;  however,  in  the 
following  section  the  discussions  and  implications  for  theory  indicate  ways  to  improve  the 
framework. 

Discussion,  Theory  Implications,  and  Recommendations 

The  concept  framework  for  this  study  was  developed  from  the  literature  describing 
disruptive  innovation  and  architectural  configurations.  The  extant  literature  strongly  indicated 
that  differing  technology  approaches  were  deliberate  choices  that  manifested  themselves  in 
the  architecture  of  technology  solutions  (Brusoni  &  Prencipe,  2011;  R.  Henderson,  1993;  R. 
M.  Henderson  &  Clark,  1990).  This  was  partially  borne  out  by  the  study  results.  The  survey 
instrument  was  able  to  reliably  identify  a  sustaining  TDA  among  the  companies  sampled. 

Analyses  of  the  data  also  successfully  quantified  the  relative  contributions  of  TDA, 
inventiveness,  experience,  capacity,  and  focus  on  winning  SBIR  Phase  1  contributions.  The 
results  allow  fruitful  discussion  on  the  disruptive  innovation  theory,  defense  acquisition,  and 
technology  development  strategies  in  small  companies,  the  concept  framework  used  in  this 
study,  and  the  characterization  of  technology  development  itself. 

Characterizing  TDA 

A  failure  of  this  study  opened  up  an  unexpected  avenue  of  discussion  regarding  the 
nature  of  technology  development.  The  instrument  used  in  this  study  reliably  measured 
sustaining  TDA  but  failed  to  identify  a  reliable  scale  for  disruptive  TDA.  The  principal  axis 
factoring  analysis  identified  a  potential  factor  that  could  be  characterized  as  the  basis  for 
disruptive  TDA;  however,  the  component  elements  were  too  dissimilar  to  make  for  a  reliable 
construct. 

The  fact  that  the  instrument  could  discern  sustaining  TDA  and  not  find  a  disruptive 
analog  raises  the  question  of  whether  disruptive  TDA  can  be  developed  as  a  homogeneous 
construct.  The  identified  variable  sTDA  displayed  homogenous  and  strongly  related 
components,  as  anticipated  from  the  extant  literature  (Christensen,  1997;  Christensen  & 
Raynor,  2003;  R.  Henderson,  1993;  R.  M.  Henderson  &  Clark,  1990).  The  incoming  study 
assumption  was  that  disruptive  TDA  would  show  a  similar  level  of  homogeneity.  The  failure 
to  do  so  highlights  what  the  literature  usually  identifies  as  a  weakness  in  disruptive 
innovation  theory.  Instead  of  a  weakness,  the  results  may  confirm  the  heterogeneous  nature 
of  disruptive  TDA  as  an  accurate  description  of  the  construct. 

Disruptive  innovation  theory  has  been  validly  criticized  as  being  weak  in  its  predictive 
effectiveness  (Danneels,  2004,  2006;  Markides,  2006;  Sood  &  Tellis,  2011;  Tellis,  2006).  A 
measure  of  this  weakness  lies  in  the  way  the  theory  was  originally  articulated.  The  same 
terminology  describes  not  only  an  intrinsic  attribute  of  the  technology,  but  also  as  a  way  to 
describe  market  effects.  Most  of  the  extant  literature  focuses  on  market  effects  with  only  a 
few  concentrating  on  finding  ex  ante  indicators  of  disruptive  technology  (Christensen,  1997; 
Govindarajan  &  Kopalle,  2006). 

The  results  of  this  study  support  the  notion  that  disruptive  TDA  may  be  an  alternate 
TDA  that  is  compositionally  more  diverse  than  its  sustaining  TDA  analogy.  This  study 
assumed  TDA  was  a  fixed  attribute  of  a  company’s  technology  development  strategy.  With  a 
choice  of  TDA — sustaining  or  disruptive — a  company  could  use  approaches  that  best  fit 
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technology  development  requirements.  Company  development  approaches  vary  from  the 
linear  and  deliberate  to  less  structured  and  more  free-wheeling  (Chaffee,  1985;  Miles,  Snow, 
Meyer,  &  Coleman,  1978;  Snow  &  Hrebiniak,  1980).  Incorporating  lessons  learned  from  the 
market  in  developing  technology  solutions  is  a  viable  strategy  in  a  competitive  market 
(Lampel  &  Shamsie,  2003;  Mintzberg,  1979;  Mintzberg  &  McHugh,  1985;  Pascale,  1984). 
Coupled  with  the  relatively  weak  contribution  of  TDA  to  SBIR  Phase  1  success  it  could  be 
that  TDA  is  a  mediating,  rather  than  primary,  driver  of  SBIR  Phase  1  success,  and  a  tool  that 
is  a  mixture  of  disruptive  and  sustaining  approaches  to  technology  development. 

TDA  and  Defense  Acquisition 

The  study  results  would  indicate  that  sustaining  TDA  is  the  preferred  approach  of 
small  innovative  companies  developing  technology  solutions.  Such  a  conclusion  would  be 
consistent  with  the  general  direction  of  defense  technology  development  used  by  the 
industrial  base  since  the  end  of  World  War  II  (Guilmartin  Jr.,  1994;  Watts,  2008;  Watts  & 
Harrison,  2011).  Sustaining  technologies  in  defense  acquisition  have  large,  well-established 
value  networks  in  a  market  with  large  barriers  to  entry  (Christensen,  1997;  Porter,  1980; 
Porter,  2008).  It  is  also  a  market  that  tends  to  be  risk-averse  with  respect  to  launching  new 
technology  architectures  that  are  the  essence  of  disruptive  innovation,  favoring  incremental, 
albeit  expensive,  expansion  of  existing  architectures  primarily  because  of  their  existing  value 
networks  (GAO,  2006,  2006b,  2007;  Meier,  2008). 

The  notion  of  TDA  being  a  scenario-specific  tool  may  account  for  the  preponderance 
of  sustaining  TDA  in  this  research.  Defense  organizations  invest  their  research  and 
development  funds  carefully,  even  at  the  SBIR  Phase  1  level.  Companies  competing  for 
such  funds  will  be  sensitive  to  how  their  nascent  technology  may  fit  in  with  the  existing  value 
network  of  a  prospective  customer  (Audretsch,  2003;  Wallsten,  2000).  A  complementary 
study  to  this  one  would  be  to  control  for  the  wording  and  scope  of  the  SBIR  Phase  1 
solicitations  when  assessing  TDA. 

The  preference  for  sustaining  technologies  may  be  at  work  on  the  customer  side  of 
the  SBIR  Phase  1  process.  Representative  from  defense  agencies  are  used  to  assess 
Phase  1  proposals  for  feasibility.  This  assessment  may  introduce  an  element  of  selection 
bias  if  assessment  agents  are  conceptually  or  technically  invested  in  the  value  networks  of 
previously  acquired  products.  Truly  disruptive  solutions  would  likely  be  discounted  if  they 
represent  to  big  a  departure  from  the  state-of-the-practice  of  the  legacy  technology.  The 
idea  of  a  selection  bias  is  entirely  consistent  with  disruptive  innovation  theory.  Legacy 
solutions  with  known  characteristics  and  measurable  value  networks  are  attractive 
alternatives  to  the  unknowns  of  new  architectural  configurations  (Christensen,  1997; 
Christensen  &  Raynor,  2003;  Guilmartin,  1994). 

Recognizing  this  selection  bias  in  defense  acquisition  means  that  any  disruptive  TDA 
would  be  a  relatively  small  presence  related  to  sustaining  TDA.  To  offset  this  bias  the  study 
sampling  frame  was  drawn  from  DARPA  solicitations  on  the  assumption  that  the 
requirements  for  the  technology  solutions  would  be  less  defined  and  conducive  to  alternate 
technology  architectures.  Follow-on  studies  controlling  for  sampling  frame  composition  may 
be  more  effective  at  isolating  disruptive  TDA.  It  is  encouraging  that  a  factor  representing 
disruptive  TDA  could  be  isolated,  even  if  it  proved  unsuitable  for  further  analysis  in  this 
study. 

Revisiting  the  Conceptual  Framework 

This  exploratory  study  posited  a  conceptual  framework  in  Figure  1  where  TDA  was 
the  primary  driver  for  SBIR  Phase  1  success.  The  resulting  data  collection  and  analysis 
gives  an  opportunity  to  review  the  framework  and  develop  an  alternate  configuration  for 
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follow-on  analysis.  From  the  regression  analysis  it  appears  that  a  company’s  focus  on  SBIR 
projects  with  experience,  measured  in  terms  of  company  age,  a  mediating  variable. 
Inventiveness  and  sTDA  were  found  to  be  weaker  contributors,  with  capacity  contributing 
very  little. 

Two  concepts  from  the  discussion  of  this  research  suggest  a  rearrangement  of  the 
original  framework.  The  first  concept  is  where  TDA  is  actually  a  supporting  tool  that  is 
flexible  in  responding  to  the  specifics  of  a  given  solicitation.  This  moves  TDA  away  from 
being  a  primary  factor  to  being  a  mediating  variable.  The  second  concept  is  to  recognize  the 
existence  of  some  assessment  bias  on  the  part  of  organizations  in  selecting  SBIR  Phase  1 
proposals.  The  reworked  conceptual  framework  is  found  at  Figure  3. 


Figure  3.  Modified  Conceptual  Framework 

Focus  is  now  the  conceptual  anchor  for  the  construct  with  inventiveness, 
assessment,  and  experience  being  the  main  mediators.  The  main  difference  between  this 
construct  and  the  initial  framework  is  the  roles  of  TDA  and  capacity.  They  are  now  shown  to 
be  mediating  inventiveness.  Capacity  was  only  a  weak  contribution  to  the  criterion  variable 
and  in  the  new  framework  it  is  posited  to  have  a  more  direct  relationship  to  inventiveness.  In 
a  similar  fashion,  TDA  characterized  as  a  flexible  tool  would  have  a  more  direct  effect  on 
inventiveness.  Science  and  technical  staff  would  have  the  opportunity  to  blend  disruptive 
and  sustaining  approaches  to  find  the  best  mix  in  developing  technology  solutions. 
Assessment  is  a  new,  outside  factor  added  to  control  for  the  posited  assessment  bias 
discussed  earlier. 

Recommendations  for  Future  Work 

The  following  recommendations  are  drawn  from  the  above  discussion  and  noted 
limitations  of  the  study.  They  are  structured  to  yield  a  more  effective  instrument  for 
establishing  sustaining  and  disruptive  TDA  and  to  increase  the  applicability  to  scenarios 
outside  the  defense  context. 

Enhanced  sampling.  The  results  of  this  study  are  limited  in  applicability,  in  part 
because  of  the  relatively  small  sample  of  16  respondents.  A  much  larger  sample  size  of 
100-200  responses  is  recommended,  which  would  add  more  confidence  and  precision  in 
the  dimension  reduction  step  (Costello  &  Osborne,  2005).  Increasing  the  sample  size  is 
perhaps  the  single  most  effective  measure  to  clarify  the  ambiguities  surrounding  the 
constructs  of  sustaining  and  disruptive  TDA. 
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A  follow-on  study  would  benefit  from  controlling  for  technology  domains.  While  not 
examined  in  this  study  it  is  reasonable  to  assume  that  technology  development  of  physical 
items  may  use  different  TDA  than  software.  The  sampled  companies  varied  widely  in  terms 
of  their  technology  domains.  The  new  conceptual  framework  of  Figure  3  posits  TDA  as  a 
tool  mediating  inventiveness.  A  refinement  of  this  research  would  be  to  determine  if  TDA  for 
different  technology  domains  could  be  quantified. 

Survey  instrument  enhancement.  The  survey  instrument  used  for  this  study  would 
need  to  be  revisited  for  a  study  based  on  Figure  3.  The  lessons  of  this  study,  particularly  in 
regard  to  defining  disruptive  TDA,  would  need  to  be  incorporated  to  define  a  more  reliable 
construct  and  scale  for  disruptive  TDA.  A  new  section  to  measure  assessment  bias  would 
be  needed  to  properly  integrate  the  new  assessment  construct  of  the  concept  framework. 

Exploratory/confirmatory  analysis.  A  new  exploratory  study  should  be  conducted 
with  the  new  conceptual  framework,  followed  by  a  confirmatory  study.  Both  should  be  done 
with  a  relatively  large  sample.  A  confirmatory  study  should  be  preceded  by  the  development 
of  a  quantitative  framework  through  the  use  of  structural  equation  modeling. 

Expansion  to  commercial  domain.  Successful  confirmatory  analysis  would 
enhance  the  confidence  in  the  concept  framework  to  the  point  where  generalizations  could 
be  made  about  TDA  in  larger  companies  with  a  commercial,  non-defense  focus.  In  this  final 
study  the  sampling  frame  is  unrestricted. 
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Research  Background 

•  US  Department  of  Defense  heavily  invests  in  technology 

•  Most  defense  innovations  are  derivative  ("sustaining") 

•  Sustaining  innovations  drawn  from  existing  technology  base 

•  "Disruptive"  innovations  potentially  expand  the  technology  base 

•  May  lead  to  new  and  advanced  defense  capabilities 

•  Disruptive  innovations  are  difficult  to  identify  in  advance 

•  Disruption  defined  by  technology  attributes  and  market  effect 

•  Market  effect:  a  disrupted  relationship  among  competitors 

•  Technology  attribute:  existing  technology  in  new  architectural 
configurations 

•  New  configurations  imply  deliberate  technology  development  approach 
(TDA) 

•  Does  knowing  a  company's  TDA  help  with  predicting  disruptive 
innovations? 
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Methodology 

•  Non-experimental,  quantitative,  correlational  research  design 

•  Posit  a  relationship  among  TDA,  company  demographics,  and  winning  SBIR 
Phase  1  contracts 

•  Survey  SBIR  participants  to  gather  required  information 

•  Small  innovative  companies  developing  new  technologies 

•  DARPA  SBIR  Phase  1  awardees  chosen  as  surrogate  population 

•  New  survey  instrument  gathers  primary  data: 

•  Company  demographic  data  (focus,  inventiveness,  experience,  capacity) 

•  TDA  data  (Likert-style  responses  to  development  preferences) 

•  Exploratory  Factor  Analysis  (EFA)  used  to  develop  scales  for  sustaining  and 
disruptive  TDA 

•  TDA  score  calculated  for  each  company 

•  TDA  values  used  as  independent  variable  in  follow-on  analyses 

•  Determine  relationships 

•  Correlation  analysis 

•  Regression  analysis 


Initial  Concept  Diagram 
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Research  Questions  &  Hypotheses 

•  Research  Questions 

•  RQl  What  is  the  relationship  among  the  independent  variables  of  TDA, 
company  demographics,  and  the  criterion  variable  of  winning  Phase  1  SBIR 
defense  contracts? 

•  RQ2  What  are  the  relative  contributions  of  TDA,  and  company  demographics 
to  winning  SBIR  Phase  1  contracts? 

•  Research  Hypotheses 

•  H01  There  is  no  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

•  Hjl  There  is  a  significant  difference  between  technology  development 
approaches  in  winning  SBIR  Phase  1  defense  contracts. 

•  H02  There  is  no  significant  relationship  between  TDA  and  the  contribution  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts. 

•  H12  There  is  a  significant  relationship  between  TDA  and  the  contributions  of 
the  demographics  INVENTION,  CAPACITY,  FOCUS,  and  EXPERIENCE  of 
companies  who  win  SBIR  Phase  1  defense  contracts. 
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TDA:  Measuring  a  New  Construct 

•  TDA  is  a  new  construct  for  measuring  preference  for  architectural  re¬ 
configuration 

•  Existing  surveys  do  not  adequately  measure  TDA 

•  New  instrument  required 

•  New  instrument  based  on  disruptive  innovation  and  systems 
engineering  theory 

•  12  Likert-style  questions  developed  from  disruptive  innovation  theory 

•  Example:  "Your  company's  technology  development  approach  is  based  on 
improving  component  technology  in  existing  system  architectures" 

•  Measures  an  organization's  preferred  strategy  for  developing 
technology  solutions: 

•  Disruptive  =>  architectural  re-configuration  with  extant  technology 

•  Sustaining  =>  incremental  improvement  along  existing  technology  trajectory 

•  EFA  used  to  determine  sustaining  and  disruptive  factors 

•  Develop  scales  for  measuring  disruptive  and  sustaining  TDA 

•  Use  measures  of  TDA  in  correlation  and  regression  analyses 


Population  and  Sampling 


•  Target  Population 

•  US  companies  of  employing  <  500 

•  Engaged  primarily  in  technology  development 

•  Desire  to  contract  with  US  Department  of  Defense 

•  Sampling  Frame 

•  Companies  receiving  a  DARPA  SBIR  Phase  1  contract 

•  Phase  1  awards  from  2007  -  2013 

•  462  companies  and  over  600  Phase  1  Awards 

•  DARPA  solicitations  relatively  open-ended 

•  Covers  a  wide  variety  of  issues  and  invites  innovative  solutions 

•  Sufficient  number  of  companies  to  conduct  pilot  and  full-scale  surveys 


•  Sampling  Results 

•  Companies  contacted:  70 

•  Companies  expressing  interest:  36 

•  Companies  requesting  email  link:  18 

•  Usable  responses:  16 

•  Response  rate:  ~23% 
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Establishing  TDA  -  Factor  Extraction 


Scree  Plot 
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Establishing  TDA-  Factor  Loading  &  Interpretation 


Factor 

1 

2 

3 

4 

TDA16  .813 

.118 

-.074 

1537 

TDA21  .774 

-.139 

-.236 

|500 

TDA17  .716 

-.243 

.206 

.037 

TDA15  .617 

-.264 

-.325 

-.238 

TDA19  .508 

-.411 

-.463 

.248 

TDA26  .725 

-.783 

-.223 

-.093 

TDA23  .363 

-.708 

-.150 

.045 

TDA22  -.017 

-.607 

-.211 

-.149 

TDA20  .024 

-.529 

.388 

-.027 

TDA24  .150 

.091 

.710 

.157 

TDA  18  -.341 

.176 

.649 

.096 

TDA25  .062 

.063 

.134 

|793 

Extraction  Method:  Principal  Axis  Factoring. 

Rotation  Method:  Oblimin  with  Kaiser  Normalization. 
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Scale  Development 

•  Developed  from  12  Likert-style  questions  on  technology 
development 

•  Four  factors  extracted;  candidate  scales  developed 

•  Factor  #1:  accurate  characterization  of  sustaining  TDA  (sTDA) 

•  Factor  #2  &  #3:  mixed  elements  of  sustaining  and  disruptive  TDA 

•  Factor  #4:  redundant  with  Factor  #1 


Factor 

Cronbach's  Alpha 

Cronbach  Alpha  Ranqe 

Item  Deletion  Sensitivity 

1 

.833 

.757  -  .832 

0 

2 

.727 

.601  -  .753 

3 

3 

.448 

.227  -  .526 

n/a 

4 

.762 

.532  -  .803 

2 
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Research  Question  1,  Hypotheses  H01  and  H11 

•  RQ1:  highly  correlated  and  significant  relationship  among 
demographics  and  SBIR  success 

•  Strong  correlations  among  demographics  and  SBIR  success 

•  Company  age,  experience,  and  focus  show  strongest  relations 

•  Weaker,  but  still  significant,  correlation  with  sTDA 

•  Study  results  unable  to  address  H01  and  H11 

•  Scale  successfully  developed  for  sustaining  TDA 

•  No  reliable  scale  developed  for  disruptive  TDA 

•  Candidate  scales  fell  short  in  two  areas: 

•  Sensitivity  to  item  deletion 

•  Contained  mixed  elements  of  disruptive  and  sustaining  TDA 
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Research  Question  2,  Hypotheses  H02  and  Hx2 

•  RQ2:  results  successfully  quantify  the  contribution  of  TDA 
and  demographic  variables  to  SBIR  Phase  1  success 

•  SBIR  Phase  1  success  regressed  against  company  demographics 
and  TDA 

•  Company  age,  focus,  inventiveness,  TDA,  and  capacity  best 
predicted  SBIR  Phase  1  success 

•  Regression  analysis  supports  rejecting  H02  and  accepting  H12 

•  ~87%  of  SBIR  success  explained  by  independent  variables 

•  Regression  model  coefficients  performed  significantly  better 
than  the  null  hypothesis  of  "0"  values  for  linear  model 
coefficients 
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Research  Observations 

•  Initial  Concept  Framework 

•  Assumed  TDA  is  primary  determinant  of  SBIR  success 

•  TDA  may  only  be  a  minor,  contributing  factor 

•  Characterizing  Disruptive  TDA 

•  Survey  instrument  reliably  characterized  sustaining  TDA 

•  Disruptive  TDA  lacked  distinct,  clear  definition 

•  Elements  of  disruptive  TDA  evident  in  all  extracted  factors 

•  Sample  Size 

•  Sample  set  of  16  responses  sufficient  for  exploratory  work 

•  Larger  samples  improve  all  aspects  of  analysis 

•  Insufficient  to  develop  a  disruptive  TDA  scale  with  existing  instrument 

•  Sample  Frame  Composition 

•  Initial  inclusion  criteria:  <500  employees,  at  least  one  SBIR  contract,  DARPA  solicitations 

•  Wide  variety  of  technology  types,  e.g.,  software,  hardware,  services 

•  Different  technology  focus  may  mean  varying  TDA 

•  External  Validity 

•  Results  may  be  unique  to  sampling  frame 

•  DARPA  solicitations  broadly  written;  encourages  wider  variety  of  technology  solutions 

•  Sustaining  TDA  may  be  more  prevalent  in  SBIR  solicitations  from  different  federal  agencies 
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Discussion  and  Theory  Implications 

•  Disruptive  Technology:  Intrinsic  Attribute  or  Market  Effect? 

•  Evidence  for  sustaining  TDA  as  an  intrinsic  corporate  attribute 

•  Homogeneous  description 

•  Easily  factored  from  survey  instrument  results 

•  Strong  loading  in  factor  interpretation  process 

•  Disruptive  TDA  -  possibly  a  blend  of  intrinsic  and  market  effects 

•  Mixed  composition  of  sustaining  and  disruptive  attributes 

•  Intrinsic  =>  part  of  a  technology  development  strategy 

•  Market  effects  =>  feedback  from  the  market  informs  development  strategy 

•  Essence  of  the  "organizational  learning"  business  strategy 

•  Better  characterization  of  disruptive  TDA  required  to  separate  intrinsic 
properties  from  market  effects 
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Discussion  and  Theory  Implications  -  cont. 

•  Role  of  TDA  in  Company  Strategy 

•  Extracted  factors  displayed  disruptive  and  sustaining  components 

•  Technology  companies  use  different  strategies  for  developing  technology 
solutions 

•  TDA  may  be  a  flexible  tool  in  a  broader  technology  development  strategy 

•  Companies  may  shift  between  sustaining  and  disruptive  approaches 
depending  on  project  requirements 

•  Ambiguous  project  requirements  =>  learning-by-doing/disruptive  TDA 

•  Clarified  project  requirements  =>  sustaining  TDA 

•  TDA  and  Bias  in  Defense  Acquisition 

•  SBIR  proposal  assessments  may  bias  toward  sustaining  TDA 

•  Assessment  process  may  not  recognize  disruptive  TDA 

•  Sustaining  technologies  dominate  in  defense  acquisition 

•  Proposal  assessments  focused  on  most  familiar  technology  solutions 
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Modified  Concept  Diagram 


INVENTION 


EXPERIENCE 


ASSESSMENT 
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Recommendations  For  Future  Research 

•  Increase  Sample  Size 

•  Increase  sample  size  to  ~200  company  responses 

•  Decrease  ambiguity  regarding  disruptive  TDA;  increase  precision 

•  Increase  reliability  of  regression  analysis 

•  Controlled  Sample  Frame  Composition 

•  Sample  frame  included  a  variety  of  technologies 

•  Control  for  different  technology  solutions,  e.g.,  hardware  vs.  software 

•  Achieved  through  study  inclusion  criteria  or  in  regression  analysis 

•  Survey  Instrument  Enhancement 

•  Incorporate  results  from  increasing  sample  size  and  controlling  sample  frame  composition 

•  Compare  with  original  instrument 

•  Enhanced  Concept  Framework 

•  Modify  framework  to  align  with  previous  research  results 

•  Shift  from  exploratory  to  confirmatory  analysis 

•  Use  structural  equation  modeling  to  quantify  relationships 

•  Expansion  to  Commercial  Domain 

•  Generalize  the  enhanced  framework  to  non-defense  companies 

•  Test  the  framework  in  a  larger  environment 


